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Distributed  Mission  Training  (DMT)  is  a  revolutionary  team-training  para- 
digm  currendy  evolving  at  the  U.S.  Department  of  Defense,  especially  in  the  Air  Force. 
The  objective  of  DMT  is  to  concurrently  train  people  in  team  efforts  involving  coordina¬ 


tion,  communication,  and  decision  making.  The  teams  may  not  necessarily  be  co-located 
and  could  be  engaged  in  independent  as  well  as  coordinated  tasks  at  remote  sites. 


While  the  U.S.  is  pioneering  these  technolo¬ 
gies,  several  other  countries  such  as  U.K.,  France, 
Israel,  and  the  Netherlands  are  also  seriously 
engaged  in  such  initiatives. 

The  rapid  expansions  in  the  dimensions  and 
complexity  of  contemporary  team  missions  are 
leading  to  a  tremendous  emphasis  on  the  needs 
for  effective,  integrated  training  systems  to  pro¬ 
vide  realistic  environments  for  the  acquisition  of 
sophisticated  and  multifaceted  team  skills.  How¬ 


ever,  such  training  has  yet  to  come  of  age.  One  of 
the  primary  reasons  is  increasing  resource  limita¬ 
tions  such  as  the  availability  of  actual  equipment 
for  training  purposes,  costs  associated  with  real- 
world  training  environments,  and  safety  and 
security  reasons  that  could  forbid  mission  train¬ 
ing  in  real  operating  scenarios.  This  predicament 
severely  restricts  the  scope  of  live,  integrated 
team-training  possibilities.  Also,  team  efforts 
!  could  involve  several  geographically  dispersed 
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#DMT  IS  STILL  VERY  MUCH  THE  STATE-OF- 

THE-ART.  It  is  expected  to  become  the  state-of- 
practice  in  the  next  millennium. 
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individuals  operating  from  different  platforms  at 
remote  sites,  who  need  to  work  together  in  a  coordi¬ 
nated  manner  to  achieve  the  mission  objectives. 
From  a  practical  standpoint,  it  could  be  very  diffi¬ 
cult,  or  even  impossible,  to  assemble  them  in  a  sin¬ 
gle  place  to  train  together. 

These  constraints  are  especially  evident  in  mili¬ 
tary  training  programs  that  require  multiple  air 
crews  operating  diverse  units  such  as  aircraft  and 
other  weapon  systems  in  different  places.  They  need 
to  train,  communicate,  coordinate,  and  work 
together  to  achieve  common  objectives.  Until 
recently,  such  training  was  usually  conducted  in  iso¬ 
lated  programs  at  different  bases  using  both  the 
actual  equipment  and  simulated  environments. 
Consequently,  the  full  potential  of  integrated  team 
training  has  yet  to  be  realized.  In  fact,  it  is  some¬ 
what  paradoxical  that  while  people  are  required  to 
operate  as  integrated  teams  in  real-world  situations, 
the  constraints  in  training  do  not  allow  them  to 
practice  such  missions  beforehand  and  prepare  for 
eventual  demands. 

The  answers  to  these  daunting  challenges  lie  in 
combining  three  evolving  streams  of  technology: 


Vi?,  remote  networking,  and  multi¬ 
media  (image,  data,  voice,  video) 
communication — the  backbone  of 
DMT.  While  many  useful  VR 
applications  are  fast  emerging  in 
diverse  areas  such  as  medicine, 
engineering  design,  scientific 
visualization,  and  education, 
human  training  is  a  fundamental 
area  where  it  is  critically  needed. 
Indeed,  it  has  already  made  a  sig¬ 
nificant  impact.  Training  teams 
of  people  to  coordinate  and  work 
together  toward  a  common  objec¬ 
tive  is  a  big  challenge.  VR,  cou¬ 
pled  with  powerful  image 
processing  and  distributed  net¬ 
working  technologies,  presents  a 
formidable  array  of  technological 
solutions  to  this  challenge.  DMT 
is  a  concept  that  originated  from 
this  perspective,  and  has  evolved 
into  a  prototypical  technology  level  that  leads  to 
experimentation,  analysis,  and  further  development. 

DMT  is  based  on  three  principles:  teams — not 
individuals — execute  missions,  team  skills  are  built 
upon — but  different  from — individual  skills,  and  a 
combination  of  the  three  technology  streams  could 
enable  creative  training  opportunities  that  overcome 
the  limitations  of  time,  distance,  and  training 
resources.  Consequendy,  DMT  is  strongly  emerging 
as  an  effective  mode  of  team  training.  A  large  indus¬ 
trial  support  base  consisting  of  companies  engaged  in 
training  methodologies  and  systems,  virtual  simula¬ 
tion  platforms,  networking,  and  multimedia  database 
systems  is  rapidly  developing.  Additionally,  these 
efforts  are  sparking  considerable  research  on  team 
training  and  the  associated  technologies  at  universi¬ 
ties,  federal  laboratories,  and  the  industrial  sector. 

Although  DMT  as  a  concept  has  originated  from 
the  training  needs  of  the  military,  it  has  far-reaching 
implications  to  team  training  in  numerous  other 
fields  as  well.  Training  commercial  pilots,  air  traffic 
controllers,  navigators,  instrumentation  specialists, 
and  assembly-line  workers  all  require  real-time  coor¬ 
dination  and  communication.  The  DMT  concept 


66 


September  1999/Vol.  42.  No.  9  COMMUNICATIONS  OF  THE  ACM 


provides  a  powerful  framework  to  develop  integrated 
virtual,  constructive,  and  real  team-training  plat¬ 
forms  for  these  applications. 

DMT  is  still  very  much  the  state-of-the-art.  It  is 
expected  to  become  the  state-of-practice  in  the  next 
millennium.  Currently,  distributed  networks  linking 
several  aircraft  simulators  at  each  DMT  node  have 
been  developed  and  implemented  in  the  U.S.  Air 
Force.  They  will  be  deployed,  tested,  and  eventually 
inducted  into  their  larger  training  systems.  The  current 
DMT  trainer  web  is  a  secure  intranet  hosting  actual 
aircraft,  and  virtual  and  constructive  aircraft  simula¬ 
tions  and  other  logistics  support.  Extensive  research 
and  development  on  DMT  systems  is  an  ongoing, 
high-priority  initiative  within  the  U.S.  Air  Force. 

While  DMT  is  a  promising  technological  revolu¬ 
tion  in  the  training  world,  there  are  also  several  chal¬ 
lenges  ahead.  Some  of  the  critical  questions 
remaining  are:  How  and  when  DMT  should  be  used 
as  part  of  a  formal  training  program  involving  a  mul¬ 
titude  of  team  skills  and  equipment?  What  are  the 
technological  design  options  and  how  should  DMT 
systems  be  configured?  What  are  the  current  techno¬ 
logical  and  operational  limitations  of  the  DMT  tech¬ 
nologies,  and  how  can  they  be  overcome?  How 
should  DMT  systems  be  developed  and  implemented 
in  any  team  training  application? 

These  questions  lead  to  an  analysis  of  the  poten¬ 
tials  and  challenges  in  DMT  from  three  perspectives: 
behavioral,  technological,  and  system  implementa¬ 
tion  (see  the  accompanying  table). 

In  this  section,  we  try  to  provide  some  answers  to 
these  specific  questions.  Carroll  presents  a  vision  for 
the  emerging  DMT  landscape  and  Bell  addresses  the 
behavioral  questions.  Dahmann  et  al.  present  an 
architectural  standard  for  DMT  that  will  soon 
become  an  IEEE  and  NATO  standard.  Peppier  and 
Stephens  address  the  challenges  and  emerging  solu¬ 
tions  in  visual  imagery  in  virtual  platforms,  and 
Crane  presents  an  analysis  of  a  real-world  DMT 
implementation,  practical  experience  with  DMT,  and 
performance  assessments  from  controlled  team  train¬ 
ing  studies.  H 
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Distributed  Mission  Training  is  the  creation  of  a  shared  environment 
comprised  of  real,  virtual,  and  constructive  systems  that  allow  teams  of  individuals  to  train 
both  individually  and  collectively.  While  DMT  is  primarily  a  team-training  concept  that 
has  evolved  in  the  U.S.  Department  of  Defense  for  its  operational  forces,  its  foundations 
in  terms  of  training  principles  and  technologies  have  far-reaching  implications  to  the  field 
of  human  training  in  general. 


The  concept  of  DMT  and  its  potential  applica¬ 
tions  have  their  roots  in  the  creation  of  an  immer¬ 
sive,  fully  integrated,  seamless  information  system 
that  connects  independent  simulation-based  train¬ 
ing  environments  to  operate  together.  The  result 
is  a  synergistic,  hybrid  environment  of  VR 
systems  in  which  information  is  dynami¬ 
cally  shared  and  used  among  a  group  of 
individuals  engaged  in  real-time  mission 
critical  activities  requiring  coordination 
and  communication.  Such  a  hybrid 
environment  would  provide  significant 
economies,  compared  to  the  cost  of 
moving  people  and  resources  in  training  at 
the  scale  of  global  operations  with  actual 
equipment.  This  concept  is  particularly  relevant 
to  the  training  programs  in  the  military,  where  col¬ 
lective  training  involving  several  weapon  systems, 


|  command  and  control  systems,  and  training  sys¬ 
tems  is  becoming  increasingly  critical.  This  train¬ 
ing  possibility  is  envisioned  through  a  seamless 
integration  of  real,  virtual,  and  constructive 
domains  of  operation  in  a  global  environment. 
The  synergistic  combination  of  the  three 
primary  domains  of  DMT  can  become 
both  the  training  and  the  operational 
backbone  of  future  enterprises,  where 
actual  operations  and  training  events  are 
differentiated  only  by  their  objectives  and 
outcomes.  The  real  portion  consists  of 
the  actual  operational  equipment;  in  the 
military  context,  this  will  be  weapon  sys¬ 
tems,  command  and  control  systems,  informa¬ 
tion,  surveillance,  and  reconnaissance  assets  and 
the  logistical  support  infrastructure.  The  virtual 
portion  consists  of  the  training  environment  and 


media,  such  as  equipment  simulators,  synthetic  envi¬ 
ronments,  planning/preview/rehearsal  systems;  train¬ 
ing  management  systems,  distributed  and  secure 
connectivity,  instructor/operator  stations,  perfor¬ 
mance  measurement  systems  and  archival  capabilities. 
It’s  clearly  obvious  these  components  could  constitute 
a  VR  training  environment  in  any  operational  setting. 
The  constructive  portion  consists  of  interactive  and 
animated  computer-generated  models  that  enrich  the 


training;  individual,  procedural,  and  coordinated  team 
training  involving  high  levels  of  communication  and 
decision  making.  In  the  military  context,  this  can  be 
seen  as  a  real  component  of  DMT  consisting  of 
weapon  systems  to  operate  in  existing  airspaces  and 
ranges;  a  virtual  component  consisting  of  simulators 
conduct  training  in  realistic  synthetic  renditions  of 
those  same  areas;  constructive  computer-driven  mod¬ 
els  will  provide  the  training  scenarios  at  both  the  real 


and  virtual  levels.  Real-time  net¬ 
works  provide  the  connectivity  for 
planning  and  execution  of  tasks  by 
players  at  remote  locations. 
Depending  on  their  objectives,  the 
players  will  then  have  the  option  to 
operate  using  their  respective  actual 
equipment  or  simulation  compo¬ 
nents  to  optimize  performance  and 
training  effectiveness,  and  minimize 
operational  risks  in  live  training. 

Performance  and  Cost 
Implications 

The  underpinnings  of  these  new 
training  environments  lie  in  the 
potential  of  affordable  and  effec¬ 
tive  computational,  network,  and 
simulation  technologies.  A  suite  of 
core  training  technologies  is  ren¬ 
dering  this  possible.  Regardless  of 
the  application,  the  focus  of  the 
emerging  technologies  is  to  pro¬ 
vide  individual  and  team  access  to 
mission-critical  data,  accuracy  in 
VR  representations  of  the  real 
world,  and  coordination  mecha¬ 
nisms.  With  the  increasing  needs 
to  enhance  the  quality  of  commu¬ 
nication  and  decision  making  in 
team  training,  DMT  systems  are 
beginning  to  provide  novel  capa¬ 
bilities  for  experimentation  and 
learning.  The  constructive  models 
of  simulation,  cue  coordination 


training  environment  by  adding  desired  levels  of 
operational  characteristics  and  complexity.  A  high- 
level  view  of  the  evolving  DMT  framework  is  shown 
in  Figure  1. 

The  training  and  readiness  benefits  of  DMT  will 
become  apparent  when  these  disparate  systems  are 
integrated  to  create  an  interactive,  dynamic  environ¬ 
ment.  This  will  allow  each  media  to  be  used  indepen¬ 
dently  or  synergistically  to  support  all  levels  of 


through  real-time  synchronized 
imaging  systems,  and  online  connectivity  for  both 
human  and  model-level  communications  have  been 
proven  prototypically. 

A  global  training  enterprise  like  DMT  could  be 
somewhat  expensive.  In  particular,  R&D  and  pro¬ 
curement  represent  significant  investments.  However, 
the  costs  for  global  DMT  should  not  be  viewed  as 
totally  additive.  In  fact,  if  DMT  is  viewed  as  a  unify¬ 
ing,  integrating  concept,  then  significant  economies 
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of  scale  and  scope  can  result  from  up-ffont  commit¬ 
ments  to  universally  support  its  implementation. 
Note  that  DMT  is  viewed  as  an  environment  inte¬ 
grating  real,  virtual,  and  constructive  domains.  There¬ 
fore,  the  already  committed  investments  in  the  real 
components  of  existing  training  systems  do  not 
accrue  in  DMT  costs  in  totality,  except  for  any  reengi¬ 
neering  and  adaptations  to  these  systems  that  may  be 
necessary  for  their  integration  with  the  DMT  envi¬ 
ronment.  In  the  military  context,  the  exisung  weapon 
systems  and  command  and  control  systems  belong  to 
this  category.  However,  DMT  requires  additional 
investments  in  simulators,  training  platforms,  imag¬ 
ing  systems  and  networking  capabilities.  In  fact, 
DMT  can  also  be  viewed  as  the  next  generation  vir¬ 
tual  and  constructive  training  systems  by  many  orga¬ 
nizations  when  the  existing  facilities  come  up  for 
upgrades.  Consequendy,  this  could  also  entail  a  pro¬ 
ration  and  absorption  of  some  of  the  DMT  costs 
from  existing  training  capabilities. 

Furthermore,  the  core  technologies  of  DMT  are 
nearly  universal;  they  are  common  to  most  types  of 
training,  education,  and  even  entertainment  environ¬ 
ments.  As  a  result,  multiple  benefits  in  different 
domains  of  applications  can  be  derived  from  any 
R&D  investment  in  DMT  technologies.  Reducing 


dependence  on  the  actual  operating  equipment  for 
training  by  using  DMT  in  conjunction  will  dramati¬ 
cally  lower  operating  costs  while  extending  the  lives  of 
the  actual  operating  systems.  Synergistic,  hybrid  envi¬ 
ronments  could  also  make  training  more  effective. 
DMT  can  create  a  substantial  improvement  over  cur¬ 
rent  capabilities  for  a  modest  increase  in  cost  over 
what  is  already  being  committed  for  disparate  activi¬ 
ties.  In  fact,  a  common  global  vision  for  this  integrat¬ 
ing  concept  could  conceivably  lower  the  cost  for  a  far 
superior  training  capability.  □ 
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TIT 'C  Synthetic  training  environments 

XXL  can  enhance  teamwork  performance. 

EFFECTIVENESS 

OF  DISTRIBUTED  MISSION  TRAINING 

The  advances  in  computing  and  communication  technologies 
have  led  to  synthetic  training  environments  with  tremendous  poten¬ 
tial.  Distributed  Mission  Training  is  such  an  environment.  DMT  has 
adaptively  focused  and  integrated  VR  and  networking  technologies 
for  team  training  in  dynamic  operational  environments.  Intra-  and 
interteam  communication,  coordination,  and  decision  making  in  such  envi¬ 
ronments  are  central  to  mission-critical  team  performance.  While  these  attributes  are 
especially  significant  in  military  operations,  they  define  effective  teamwork  in  general, 
ranging  from  sports  and  entertainment  to  production  and  service  environments. 


The  concept  of  DMT  has  been  technologically 
demonstrated  in  defense  operations.  Equipped 
with  advanced  image  generation  technologies, 
high-resolution  displays,  and  secure  distributed 
networks,  DMT  systems  connect  a  wide  variety  of 
local  and  geographically  dispersed  virtual  training 
platforms  for  mission-critical  team  training.  Fur¬ 
ther,  DMT  technologies  provide  extremely  high 
levels  of  both  physical  and  functional  fidelity  in 
team  training.  The  resulting  real-time  virtual  train¬ 
ing  networks  include  aircraft  simulations  and 
other  simulated  systems  such  as  tanks  and  ships.  In 


addition,  these  platforms  can  be  linked  to  actual 
training  equipment,  such  as  aircraft,  thus  offering 
the  potential  for  synthetic  environments  that  sup¬ 
port  training  at  the  individual,  team,  and  joint  ser¬ 
vice  levels.  As  a  result  of  these  increased 
capabilities,  combined  with  significant  reductions 
in  costs,  the  training  and  technology  development 
communities  are  greatly  expanding  their  use  of 
synthetic  virtual  training  environments. 

However,  there  are  still  formidable  challenges  in 
the  implementation  of  these  systems.  Indeed, 
training  is  substantially  more  than  putting  hard- 
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Table  3.  Major  skill  areas  in  effective  team  training. 


ware  and  software  systems  to  work  together.  It 
involves  managing  the  participants’  training  experi¬ 
ences  to  give  them  a  greater  potential  for  accomplish¬ 
ing  real-life  missions  than  before.  Indeed,  training  is  a 
systemic  phenomenon.  The  development  and  imple¬ 
mentation  of  instructional  systems  [4]  follows  the 
general  principles  of  systems  engineering  in  that  it 
involves  mission  analysis,  identification  of  system 
inputs  and  outputs,  and  allocation  of  functions  to  var¬ 
ious  system  components.  Based  on  this  concept,  sim¬ 
ulators  are  merely  subsystems  or  components  of  the 
overall  training  system.  Consequendy,  it  is  necessary 
to  ask  whether  or  not  these  components  are  function¬ 
ing  correcdy  and  generating  the  proper  outputs.  This 
leads  us  to  the  key  question  in  the  design  and  imple¬ 


mentation  of  DMT:  Does  the 
integration  of  DMT  technologies 
into  a  training  system  materially 
improve  the  likelihood  those  who 
use  DMT  will  successfully 
accomplish  their  missions? 

This  article  examines  this  ques¬ 
tion.  In  particular,  the  article 
focuses  on  the  challenges  DMT 
should  address,  existing  empirical 
and  analytical  data  that  gives 
cause  for  optimism,  some  design 
considerations  for  DMT  systems, 
and  the  major  cultural  changes 
that  must  take  place  in  the  way 
organizations  view  training  so 
that  DMT  can  have  the  largest 
possible  impact. 

Team  Training  Poses 
A  Naturalistic  Paradigm  • 

A  naturalistic  perspective  on 
training  is  to  view  the  process 
and  its  effectiveness  from  the 
way  people  use  their  experience 
to  analyze,  interact,  and  make 
decisions  in  field  settings  [7]. 
This  is  dramatically  different 
from  the  traditional  ways  of 
analysis  in  the  sense  that  empha¬ 
sis  is  placed  on  how  trainees  are 
concerned  about  sizing  up  a  sit¬ 
uation  and  refreshing  their  situ¬ 
ational  awareness  through 
feedback,  rather  than  reacting  to 
environmental  stimuli  and  their 
demands.  The  contextual  factors 
that  affect  the  way  real-world 
operations  occur  are  central  to 
naturalistic  analysis,  and  are  summarized  in  Table  1 . 

A  central  theme  emerging  from  an  analysis  of  train¬ 
ing  processes  is  that  decision  making  stems  from  situ¬ 
ational  awareness  and  assessment,  prioritization  in 
dynamic  task  environments,  and  action/feedback 
structures  in  event  management.  Individuals  operat¬ 
ing  collectively  in  mission-critical,  task-oriented  envi¬ 
ronments  are,  in  essence,  decision  makers  who 
together  determine  the  final  outcome  of  a  mission. 
So,  the  naturalistic  perspective  requires  an  investiga¬ 
tion  of  what  makes  them  effective  decision  makers. 
Once  established,  these  attributes  can  be  used  to  spec¬ 
ify  which  knowledge,  skills,  and  processes  must  be 
learned  in  order  to  achieve  targeted  performance. 
Table  2  presents  a  summary  of  attributes  of  an  effec- 
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Does  the  integration  of  DMT  technologies 
into  a  training  system  materially  improve  the 

likelihood  THOSE  WHO  USE  DMT  WILL 
SUCCESSFULLY  ACCOMPLISH  THEIR  MISSIONS? 


tive  decision  maker  operating  within  the  context  of  a 
team  and  its  goals. 

The  objective  of  any  team-training  enterprise  is  to 
develop  individuals  trained  to  a  desirable  level  of 
expertise  in  these  attributes.  This  translates  to  a  train¬ 
ing  focus  on  the  major  areas  shown  in  Table  3. 

This  analysis  clearly  describes  what  is  required  to 
accomplish  mission-critical  team  performance.  First, 
context-specific  domain  knowledge  is  crucial.  Next,  a 
set  of  cognitive  processes  and  skills  need  to  be 
addressed  (see  Table  3).  Finally,  the  psychomotor 
skills  required  in  the  operational  settings  need  to  be 
emphasized.  Focusing  on  these  facets  in  a  training 
program  would  accelerate  the  acquisition  of  profi¬ 
ciency,  and  the  learning  and  organizing  of  domain 
knowledge  that  supports  complex  team  maneuvers. 
These  capabilities  are  crucial  in  achieving  the  ultimate 
objective  of  a  desired  level  of  expertise  in  both  indi¬ 
vidual  tasks  and  team  missions.  Team  training  based 
on  a  combination  VR  technologies  and  real-time  con¬ 
current  connectivity  among  remote  players  addresses 
these  facets,  while  at  the  same  time  it  appears  to  over¬ 
come  most  of  the  practical  limitations  of  time  and 
space  in  a  meaningful  training  environment.  While 
concurrent  connectivity  is  necessary  in  team-training 
scenarios,  the  role  of  VR  as  a  training  medium  has 
been  a  subject  of  great  discussion  among  researchers. 
We  take  the  position  that  VR  is  indeed  an  effective 
medium  given  the  following  line  of  reasoning. 

First,  simulated  training  environments  can  signifi¬ 
cantly  accelerate  proficiency  by  exposing  trainees  to 
the  kind  of  situations  they  are  likely  to  encounter  in 
the  real  world,  but  which  could  be  hazardous  or  very 
expensive  to  practice  in  actual  operational  settings. 
For  example,  in  training  Air  Force  pilots  in  mission 
combat,  shooting  an  enemy  aircraft  can  never  be 
practiced  with  real  aircraft  exactly  as  would  be 
encountered  in  an  actual  situation.  Second,  simula¬ 
tions  can  be  controlled— the  characteristics  of  the 
training  scenarios,  situational  cues,  and  decision  out¬ 
comes  can  be  provided  as  aids  in  the  development  of 
situational  awareness,  pattern  recognition,  and  tem¬ 
plate  building.  The  constructive  models  that  play  a 
crucial  role  in  DMT  systems  are  intended  for  this 
purpose.  Finally,  simulations  are  also  an  effective 


means  to  train  reasoning  skills,  metacognitive  skills, 
risk-assessment  skills,  and  communication  skills  with¬ 
out  the  overhead  and  complexities  of  real-world 
training. 

This  analysis,  while  highlighting  the  needs  of  team 
training  from  a  naturalistic  perspective,  also  raises 
research  questions  regarding  the  training  effectiveness 
of  concurrently  connected  VR  environments  like 
DMT: 

•  On  what  basis  do  trainees  perceive  similarity  in 
training  situations?  What  triggers  a  template  in 
human  memory?  How  do  people  seek  additional 
information,  learn  from  other  co-trainees,  or 
exchange  data? 

•  How  can  the  virtual  training  components  be  inte¬ 
grated  with  real  aircraft  training  components  to 
provide  comprehensive  training  effectiveness? 

•  In  knowledge-rich  training  environments,  how 
should  the  knowledge  be  organized  into  virtual 
training  modules  so  it  fosters  ready  access  to 
information  when  necessary? 

•  How  can  the  quality  of  training  in  virtual  plat¬ 
forms  be  evaluated?  For  example,  how  do  we 
know  when  someone  becomes  an  expert? 

These  are  open  questions,  yielding  a  rich  research 
agenda. 

Lessons  from  History 

Although  there  have  been  a  number  of  efforts  involv¬ 
ing  large-scale  simulation  for  analysis,  development, 
and  training  [8],  most  of  this  early  research  focused 
on  command  and  control  systems.  Recently,  there 
has  been  a  substantial  research  thrust  on  identifying 
team  skills  and  developing  techniques  for  training 
those  skills  [10,  1 1].  These  works  have  identified  sev¬ 
eral  training  strategies  such  as  guided  practice  and 
cross-training  that  can  be  effectively  exploited  in 
DMT  systems.  However,  very  little  of  this  research 
has  been  applied  to  designing  training  for  the  high- 
performance  team  skills  typical  of  DMT. 

The  earliest  virtual  platform  training  programs 
involved  flight  simulators  and  concentrated  mainly 
on  training  basic  procedural  and  psychomotor  skills. 
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The  growth  in  computer  and  communication 
technologies  provides  unparalleled  opportunities  to 

CREATE  SYNTHETIC  ENVIRONMENTS  TO 
REVOLUTIONIZE  TRAINING. 


By  the  mid-1980s,  however,  technology  advanced  to 
the  point  it  was  possible  to  interconnect  simulators 
and  conduct  team  training.  The  first  example  of  such 
training  was  the  SIMNET  project  in  which  a  number 
of  tank  simulators  were  interconnected  to  provide 
collective  training  [1].  Subsequendy,  several  studies 
on  team-training  pilots  on  virtual  platforms  have 
been  carried  out.  Principal  among  these  were  net¬ 
worked  F-15  virtual  platforms  studies  at  McDonald 
Aircraft  Company,  Multiservice  Distributed  Training 
Testbed  (MDT2)  [5],  and  the  RoadRunner’98  study 
of  networked  DMT  systems  (see  the  Crane  article  in 
this  section).  In  all  these  studies,  a  full  spectrum  of 
simulated  team  missions  was  carried  out.  Teams  per¬ 
formed  their  normal  mission  planning  and  post-mis¬ 
sion  analyses.  A  qualified  instructor  monitored  the 
training  scenarios  and  provided  additional  guidance 
to  the  teams. 

Training  effectiveness  research  data  collected  from 
these  experiments  examined  both  process  and  prod¬ 
uct  measures.  Process  measures  pertain  to  the  fre¬ 
quency  and  quality  of  communication,  mutual 
support,  and  decision  making.  Product  measures  per¬ 
tain  to  numerical  assessments  of  performance.  In 
addition,  each  pilot  underwent  extensive  interviews 
to  determine  how  well  they  thought  the  training 
experience  prepared  them  compared  to  their  normal 
team  training  in  their  actual  equipment.  The  results 
were  quite  positive.  In  all  cases,  the  pilots  showed 
considerable  improvement  on  both  the  process  and 
product  measures  from  the  first  day  of  training  to  the 
last  day.  The  probability  of  successfully  completing  a 
mission  improved  significandy  during  the  training 
week.  Perhaps  even  more  encouraging  were  the  inter¬ 
views  that  showed  participants  were  generally  quite 
positive  about  the  experience. 

It  is  important  to  stress  how  seldom  these  teams  are 
able  to  not  only  train  together  on  a  consistent  basis 
but  also  be  able  to  plan  and  analyze  missions  together. 
When  training  with  actual  equipment,  the  physical 
distances  that  separate  individual  players  when  the 
exercise  is  over  make  it  difficult  to  have  such  sessions. 
As  a  result,  even  though  each  of  the  individual  service 
teams  was  proficient  at  its  part  of  the  mission,  they 


had  very  little  opportunity  to  integrate  their  skills 
with  the  other  members  and  work  those  skills  in  a 
realistic  tactical  environment.  All  these  studies  show  it 
is  possible  to  cross  train  using  DMT  with  multiple 
virtual  platforms  in  a  single  training  package  linked 
over  distances  to  produce  an  effective  composite 
training  environment. 

Naturalistic  Design  Considerations 

The  studies  discussed  here  indicate  that  DMT  offers 
the  potential  to  be  a  significant  training  tool.  Based 
on  the  data  gathered  thus  far,  and  faced  with  the 
increasing  pressure  to  reduce  training  costs  and 
improve  trainee  skills,  DMT  appears  to  be  a  cost- 
effective  training  approach.  Networkable,  high- 
fidelity  DMT  systems  encompassing  a  wide  range  of 
virtual  training  platforms  are  fast  emerging,  and  are 
most  likely  to  dominate  the  training  landscape  of 
the  next  millennium.  However  despite  the  current 
enthusiasm  for  DMT,  a  variety  of  significant  chal¬ 
lenges  in  achieving  the  desired  training  effectiveness 
still  remain.  We  examine  some  of  these  issues  from 
the  point  of  view  of  human-centric  and  naturalistic 
systems  design  as  follows. 

Skill  acquisition,  retention,  and  transfer.  Clearly  the 
intent  in  developing  DMT  systems  is  to  provide  addi¬ 
tional  opportunities  for  individuals  to  acquire  and 
maintain  job-relevant  sldlls.  There  is  a  great  deal  of 
basic  research  on  skill  acquisition  and  decay  that  can 
be  used  as  a  starting  point  for  identifying  those  factors 
that  influence  skill  acquisition  and  retention  [9]. 
There  is,  however,  very  little  data  describing  how  com¬ 
plex  skills  are  acquired  and  maintained  in  real-world 
work  settings  [12].  Further,  almost  no  research  involv¬ 
ing  a  detailed  examination  of  how  continuation  or 
field  training  experiences,  as  opposed  to  formal  insti¬ 
tutional  or  academic  training,  affects  the  development 
of  a  skilled  team  player.  A  well-structured  plan  of 
research  is  required  to  understand  the  development 
and  retention  of  high-performance  skills.  The  results 
of  such  research  could  then  be  used  to  define  the  train¬ 
ing  strategies  and  types  of  experiences  necessary  to 
maintain  the  training  readiness  of  teams  in  general. 

There  is  very  little  evidence  regarding  the  transfer 
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Table  4.  Critical  questions  in  developing  training  systems. 
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of  skills  from  virtual  platforms  to 
the  actual  equipment  [2]. 

Although  significant  improve¬ 
ments  have  been  repeatedly 
claimed  using  both  outcome  and 
process  measures,  almost  no 
empirical  evidence  is  available 
regarding  the  degree  to  which  the 
knowledge,  attitudes,  and  skills 
learned  or  enhanced  in  virtual 
platforms  transfer  to  actual  mis¬ 
sion  performance.  This  absence 
of  transfer  data  is  not  unique  to 
DMT.  Goldstein  [6]  observes 
that  very  few  training  programs 
include  a  systematic  evaluation  of 
their  effectiveness.  Such  evalua¬ 
tions  are  usually  difficult  to  con¬ 
duct.  Bolcovici  [3]  describes 
some  of  the  factors  and  the  prob¬ 
lems  involved  in  attempting 
transfer  of  training  studies  within 
the  military. 

Although  transfer  of  training  research  is  difficult  to 
accomplish,  it  is  essential  to  validate  the  benefits 
derived  from  DMT.  Specific  studies  on  transfer  of 
training  from  DMT  systems  to  real-world  perfor¬ 
mance  are  needed.  Without  appropriate  transfer  of 
training  data,  we  are  left  in  a  situation  similar  to  that 
encountered  by  many  college  students:  one  may  be 
capable  of  passing  the  exams  within  a  class  but  not  be 
able  to  apply  the  skills  and  knowledge  acquired  in  that 
class  in  other  courses.  In  addition,  transfer  of  training 
experiments  provide  a  means  of  determining  which 
variables  in  a  virtual  training  system  have  the  greatest 
impact  on  the  quality  of  training  yielded  by  that  sys¬ 
tem.  Such  information  is  invaluable  in  deciding 
between  alternative  system  configurations  and  devel¬ 
oping  the  most  cost-effective  training  systems.  Finally, 
critical  to  the  development  and  interpretation  of 
research  involving  skill  acquisition,  retention,  and 
transfer  is  the  need  to  develop  measures  of  individual 
and  team  performance  most  appropriate  for  the  com¬ 
plex  high-performance  skills  characteristic  of  DMT 
environments.  Without  such  measures,  it  is  difficult 
to  develop  valid  training  metrics,  validate  fidelity 
requirements,  or  determine  training  needs. 

Instructional  features.  Even  if  it  were  possible  to  cre¬ 
ate  a  synthetic  environment  that  fully  replicated  the 
real  world,  it  is  not  necessarily  the  case  that  such  a 
design  would  represent  an  optimal  training  environ¬ 
ment.  Indeed,  if  the  focus  is  on  merely  re-creating  real¬ 
ity  in  a  digital  world,  it  is  quite  possible  that  a 
suboptimal  training  environment  is  created.  Such  a 


possibility  would  result  from  failures  in  considering  the 
constraints  and  training  possibilities  of  hardware  and 
software  tools,  and  the  strategies  and  tactics  for  using 
those  tools  that  would  allow  greater  training  efficien¬ 
cies  than  are  currendy  possible  in  the  real  world. 

DMT  represents  a  quantum  leap  in  the  complexity 
of  simulation-based  training.  Indeed,  it  involves  a 
shift  from  direct  control  of  an  individual  learning  psy¬ 
chomotor  and  procedural  skills  to  indirect  control  of 
large  numbers  of  individuals  executing  complex  hier¬ 
archically  nested  sequences  of  psychomotor,  proce¬ 
dural,  cognitive,  and  team  skills  in  fluid,  rapidly 
changing  environments.  This  new  training  environ¬ 
ment,  where  the  instructor  may  be  much  more  likely 
to  be  involved  in  process  and  supervisory  control  of 
training  activities  rather  than  one-on-one  instruction, 
demands  a  human-centered  design  approach  that  tar¬ 
gets  hardware  and  software  designs  to  meet  user 
needs.  Table  4  presents  a  set  of  critical  design  ques¬ 
tions  that  must  be  well  considered  in  this  process. 

Vertical  and  horizontal  connectivity.  Training  includes 
actual  equipment,  such  as  aircraft,  operating  in  its  nat¬ 
ural  environment  (real  training  events),  simulation 
trainers  operating  in  a  virtual  environment  (virtual 
training),  and  computer-based  training  (constructive 
systems).  These  components  can  be  combined  in  a 
number  of  ways  to  create  many  different  synthetic  envi¬ 
ronments  for  training  that  range  from  a  level  of  small 
group  engagement  to  strategic  training  involving  a  joint 
task  force.  It  is  also  possible  to  nest  various  levels  of 
training  within  higher  levels.  Thus,  a  small  group  may 
be  engaged  in  a  set  of  tasks  occurring  as  a  direct  result 
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of  the  actions  taken  by  the  joint  task  force  leader  and 
the  results  of  that  engagement  will  direcdy  influence 
their  subsequent  planning  and  decisions. 

While  it  is  possible  to  link  a  wide  variety  of  live, 
virtual,  and  constructive  training  events  both  hori¬ 
zontally  across  a  capability  echelon  and  horizontally 
across  echelons,  it  is  essendal  we  identify  the  training 
benefits  for  each  level  of  participants  in  team  training. 
The  capability  to  create  larger  and  larger  scenarios 
with  more  participants  does  not  necessarily  increase 
the  training  value  of  DMT.  It  is  possible  to  position 
individuals  or  teams  in  certain  specialties  or  echelons 
serving  as  training  aids  for  other  specialties  or  eche¬ 
lons.  If  this  is  not  considered  during  the  design  of 
training  scenarios,  we  run  the  risk  of  alienating  some 
participants  and  also  reducing  their  specific  task-crit¬ 
ical  training  opportunities.  Assuming  there  is  a  valid 
training  reason  for  linking  various  echelons  of  players 
and  classes  of  training  in  mission-specific  aggregate 
training  events,  a  number  of  unresolved  technical 
issues  (for  example,  aggregation  and  separation  of 
team  members  and  communication  between  con¬ 
structive  and  manned  systems)  as  well  as  training 
issues  (for  example,  behavioral  representation  and  sce¬ 
nario  management)  still  need  to  be  examined. 

Physiological  training  issues.  In  real-life  missions, 
team  players  are  usually  exposed  to  a  wide  variety  of 
psychological  and  physiological  stresses.  While  tech¬ 
nology  is  providing  us  with  an  ever-increasing  ability 
to  replicate  many  of  the  psychological  stresses,  we  are 
still  severely  limited  in  our  ability  to  duplicate  many 
of  the  physiological  stressors  encountered  in  practice. 
For  example,  consider  the  physiological  environment 
in  which  the  modern  fighter  pilot  operates.  This  envi¬ 
ronment  involves  breathing  oxygen,  being  cramped 
in  a  small  space,  experiencing  a  number  of  conflicting 
sensor  cues,  and  enduring  various  degrees  of  G-stress. 
It  is  impossible  to  fully  replicate  these  physiological 
stressors  in  an  affordable  simulator.  Unfortunately,  we 
still  do  not  know  the  optimal  mix  of  cognitive,  pro¬ 
cedural,  psychomotor,  and  physiological  training 
required  to  develop  and  maintain  one’s  ability  to  suc¬ 
cessfully  employ  a  fast,  highly  maneuverable  fighter 
aircraft.  Althought  DMT  may  allow  trainees  to 
receive  better  training  in  certain  higher-order,  cogni¬ 
tive  tasks,  it  may  also  provide  poorer  training  in  those 
tasks  that  are  closely  tied  to  ones  ability  to  quickly 
recognize  an  opportunity  and  aggressively  maneuver 
the  aircraft  to  secure  a  tactical  advantage.  These  are 
critical  research  issues  that  need  significant  attention. 

Conclusion 

The  tremendous  growth  in  computer  and  communi¬ 
cation  technologies  provides  unparalleled  opportuni¬ 


ties  to  create  synthetic  environments  to  revolutionize 
training.  Today,  we  are  witnessing  the  first  attempts 
to  integrate  this  technology  within  Air  Force  training 
systems  as  part  of  the  DMT  program.  Initial  demon¬ 
strations  of  DMT  have  been  extremely  encouraging. 
Although  the  DMT  technologies  are  indeed  quite 
impressive,  they  represent  only  a  fraction  of  the  big 
picture  that  emerges  in  the  training  landscape. 
Equally  important  are  the  soft  technologies  (human 
factors,  education,  and  applied  cognitive  science) 
that  are  essential  for  the  development,  implementa¬ 
tion,  and  assessment  of  training.  These  soft  tech¬ 
nologies  are  critical  to  the  design  and  development  of 
training  delivery  systems  as  well  as  training  evalua¬ 
tion.  While  DMT  has  made  significant  technical 
progress,  there  still  are  a  number  of  human-centered 
challenges  that  must  be  addressed  in  order  to  deliver 
effective  and  efficient  training.  □ 
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A  novel  technology  proves  ideally  suited  to 
fast-moving  training  environs . 

IMAGE  GENERATION  SYSTEMS 

in  VIRTUAL 

TRAINING 
PLATFORMS 

Computer-image  generation  in  VR  environments  is  an  important  component  of 
Distributed  Mission  Training  systems.  In  fact,  this  is  one  of  the  limiting  factors  of  perfor¬ 
mance  that  is  technologically  challenging  in  most  synthetic  virtual  training  environments 
where  visual  cues  play  a  critical  role.  In  spite  of  this,  computer-based  training  has  signifi- 
candy  evolved  by  integrating  computer-generated  imagery  at  all  levels  of  the  education 
process.  In  virtual  training  environments,  image  generation  systems  receive 

positional  inputs  from  a  human  or  model  opera-  flight  simulators,  the  fields  of  view  are  usually 
tor,  compute  and  display  a  perspective  image  of  large,  change  at  variable  rates  depending  on  alti- 
a  3D  database  on  image  screens,  and  the  opera-  tude  and  position,  and  require  images  of  different 
tors  use  these  visual  cues  and  references  for  fur-  levels  of  clarity.  Comparatively,  in  slower  envi- 
ther  action.  Three  principal  issues  arise  in  this  ronments  such  as  automobiles  and  other  ground- 
process:  image  complexity  in  terms  of  the  size  of  based  motion  devices,  the  fields  of  view  are 
data  sets  involved,  image  clarity  (both  desired  smaller,  change  at  slower  rates,  and  require 
and  currendy  possible),  and  the  speed  of  compu-  images  of  high  clarity.  The  fields  of  view  and 
tation  and  display.  required  clarity  define  the  level  of  detail  in  the 

In  fast-moving  training  environments  such  as  terrain  involved,  and  the  consequential  database 
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Figure  I.  Conceptual  architecture  of  IG  systems. 


size;  the  rates  of  change  determine  the  computation 
and  display  speeds  required.  There  are  clear  tradeoffs 
along  these  dimensions  in  the  choice  of  an  appropri¬ 
ate  image-processing  system  besides  the  cost.  In  this 
article,  we  present  the  configuration  of  a  novel  image 
generation  (IG)  technology  for  fast-moving  training 
environments  developed  and  implemented  in  a 
DMT  prototype.  We  also  discuss  some  of  the  chal¬ 
lenges  in  the  next  stages  of  evolution  of  this  technol¬ 
ogy,  and  highlight  critical  R&D  frontiers  in  the  wide 
range  of  virtual  training  applications  of  DMT  where 
visual  systems  are  crucial. 

An  IG  system  is  usually  driven  on  the  backend  by 
an  IG  database  server  with  frontend  geometric  and 
display  processors.  The  backend  processor  is  a  critical 
determinant  of  overall  system  performance  since  it  is 
responsible  for  maintaining  very  large  image  data¬ 
bases  and  fast  information  retrieval  especially  in  high¬ 
speed  training  platforms.  The  geometric  processor 
converts  3D  database  elements  retrieved  by  the  back¬ 
end  into  2D  perspective  imagery.  (A  conceptual 
architecture  of  an  IG  system  is  shown  in  Figure  1 .) 

The  computational  speed  of  this  processor  trans¬ 
lates  to  the  latency  in  image  updates  in  the  visual  sys¬ 
tem  of  the  training  platform.  In  fast-action 
platforms,  this  speed  is  critical,  and  the  technology 
of  geometric  processors  has  considerably  improved 
over  the  last  decade  in  this  regard.  The  realism  in  vir¬ 
tual  training  platforms  is  directly  related  to  this 
latency:  the  smaller  the  latency,  the  greater  the  real¬ 
ism.  Hence,  the  quality  of  training  is  directly 
affected  by  the  speed  and  efficiency  of  the  database 
server  and  the  geometric  processor.  The  function  of 
the  display  processor  is  to  position,  align,  and  inte¬ 
grate  the  final  image  on  the  display  screens  by  map¬ 
ping  the  output  of  the  geometric  processor  into  the 


appropriate  pixels.  The  data  output  at  this  stage  is 
digital,  and  is  converted  into  analog  signals  that 
drive  the  display  devices.  The  resolution  achievable 
by  the  display  processor  and  the  end  devices  deter¬ 
mine  the  image  clarity.  They  should  be  chosen 
according  to  the  requirements  of  a  training  scenario. 

IG  applications  over  the  last  decade  have  become 
increasingly  sophisticated  with  multichannel  image- 
processing  capabilities.  In  such  systems,  multiple 
channels  of  image  development  and  displays  are  cre¬ 
ated  by  adding  more  servers  for  database,  geometric, 
and  display  processing.  While  multichannel  presen¬ 
tations  have  almost  become  the  norm  in  the  simula¬ 
tor  training  of  Air  Force  pilots,  they  are  currendy 
being  used  in  training  commercial  airline  pilots  as 
well.  The  introduction  of  the  additional  hardware  at 
the  various  stages  of  image  processing  enables  multi¬ 
ple  presentations  of  scenarios  (for  instance,  at  the 
trainee,  instructor,  and  observer  consoles),  partition¬ 
ing  the  display  field  into  screen  segments  where  each 
segment  is  uniquely  driven  by  a  chain  of  IG  proces¬ 
sors  in  order  to  lower  imaging  latencies  in  fast  train¬ 
ing  platforms,  and  the  possibility  of  creating  360° 
field  of  vision  through  multiple  projections. 

A  Current  IG  Technology 
For  Fast-Action  Platforms 

The  field-of-view,  brightness,  and  contrast  of  practi¬ 
cally  every  visual  system  available  today  is  far  less 
than  what  the  trainees  see  in  the  real  equipment  of 
fast-moving  platforms.  Most  importandy,  the  resolu¬ 
tion  of  current  display  systems  is  at  least  an  order-of- 
magnitude  less  than  that  required.  Because  of  this, 
current  visual  systems  do  not  provide  a  trainee  with 
adequate  visual  definition  to  identify  other  compo¬ 
nents  in  the  field  of  vision  in  virtual  platforms.  For 
example,  components  such  as  other  vehicles,  roads, 
and  bridges  at  realistic  ranges  need  better  visual  def¬ 
initions  for  effective  training  in  virtual  systems. 
Without  the  proper  definition  and  realism  in  the 
visual  display,  many  training  tasks  cannot  be  ade¬ 
quately  performed. 

To  solve  this  problem  for  DMT,  a  three-pronged 
approach  to  developing  advanced  visual  systems  is 
needed.  The  three  key  areas  are  low-cost,  deployable 
visual  displays  that  encompass  the  training  platforms 
and  provide  full-field-of-view  imagery;  super  high- 
resolution  display  components;  and  rapidly  gener¬ 
ated,  high-fidelity  visual  and  sensor  database 
systems.  Current  visual  systems  are  limited  by  the 
previous  generation  design  and  business  decisions. 
Although  there  is  a  significant  effort  currently  under 
way  in  the  IG  industry  to  optimize  all  the  parame¬ 
ters  available  to  current  display  technologies,  their 
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For  DMT  environments  to  meet  any  training  or 
testing  need,  the  IG  systems  must  provide  the 
visual  resolution  for  RECOGNIZING  OBJECTS 
AT  REALISTIC  RANGES  and  image  refresh  rates 
that  can  support  the  SPEED  of  fast-moving  vehicular 
training  platforms. 


fundamental  limitations  impede  their  implementa¬ 
tion  into  the  high-end  systems  of  the  future.  But 
new  technologies  are  becoming  available  that  could 
lead  to  major  advancements.  Taking  advantage  of 
these  new  technologies  and  applying  them  in  inno¬ 
vative  ways  will  be  the  key  to  producing  high-per¬ 
formance  visual  displays  of  the  future. 


Figure  2.  Architecture  of  microlaser  IG  system  and  M2DART. 


In  this  regard,  the  Air  Force  Research  Laboratory 
is  spearheading  the  development  of  an  IG  system  for 
fast-action  training  platforms.  This  system  has  been 
developed  in  conjunction  with  the  IG  industry,  and 
implemented  in  the  prototype  virtual  training  plat¬ 
forms  of  the  DMT  system  at  the  laboratory.  The  IG 
system  is  integrated  with  a  state-of-the-art  visual  dis¬ 
play  system  known  as  Mobile  Modular  Display  for 
Advanced  Research  and  Training  (M2DAR.T).  A 
high-level  architecture  of  this  IG  technology  is 
shown  in  Figure  2. 

The  new  IG  system  consists  of  a  Target  Genera¬ 


tion  Unit  (TGU)  and  Overlay  Processor  Unit  (OPU) 
in  addition  to  the  usual  components  of  an  IG  system. 
The  TGU  and  OPU  provide  necessary  inset  video 
and  background  video  format  conversion  as  well  as 
timing  synchronization.  The  OPU  converts  video 
from  an  external  background  IG  system  and  TGU 
inset  video  into  a  format  that  will  be  displayed  on  the 
special  purpose,  full  color,  super 
high-resolution,  microlaser-based 
projector  system.  A  goal  of  this 
system  is  to  resolve  20/20  visual 
acuity  area-of-interest  inset  mod¬ 
els  at  5120  x  4096  video  format. 
The  imagery  provided  by  this  IG 
is  viewed  from  the  M2DART  dis¬ 
play  system. 

M2DART  is  a  rear-screen, 
real-image,  display  system  with 
eight  flat  projection  screens  inte¬ 
grated  together  to  display  eight 
channels  of  full-color,  fiill-field- 
of-view  imagery.  Due  to  the  rela¬ 
tively  small  surface  area  on  these 
screens  in  comparison  to  large 
dome  displays,  the  imagery  is  sig¬ 
nificantly  brighter  with  much 
improved  contrast.  Our  current 
experience  with  M2DART  indi¬ 
cates  it  can  be  a  cost-effective 
solution  to  many  currently  unfulfilled  virtual  plat¬ 
form  training  requirements. 

Various  display  system  manufacturers  are  produc¬ 
ing  their  own  versions  of  the  M2DAR.T.  The  modu¬ 
larity  and  flexibility  of  the  design  of  the  M2DAR.T 
allows  it  to  remain  an  excellent  testbed  for  many 
important  technology  insertion  projects  such  as 
super  high-resolution  projection  systems,  refined 
screen  materials,  and  advanced  image  generators. 
Continued  enhancements  and  upgrades  to  the 
M2DAR.T  will  most  likely  include  modifications  to 
the  display  geometry  such  as  the  use  of  curved  sur- 
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faces  instead  of  the  current  flat-panel  facets,  and  the 
development  of  low-cost,  high-quality  collimating 
rear  projecdon  screens.  The  M2DART  appears  to  be 
a  solid  design  approach  from  which  future  DMT 
visual  displays  should  be  based. 

Challenges  and  Emerging  Solutions 

For  DMT  environments  to  meet  any  training  or 
testing  need,  the  IG  systems  must  provide  the  visual 
resolution  for  recognizing  objects  at  realistic  ranges 
and  image  refresh  rates  that  can  support  the  speed 
of  fast-moving  vehicular  training  platforms.  An 
analysis  of  M2DART  system  reveals  that  achieving 
this  level  of  acuity  requires  a  projector  system  capa¬ 
ble  of  projecting  5120  x  4096  pixels.  At  a  mini¬ 
mum,  to  produce  high-quality  dynamic  images,  the 
image  generation  update  rate  and  the  display 
refresh  rate  must  both  be  60  Hz,  noninterlaced. 
This  represents  an  order-of-magnitude  increase  over 
what  is  considered  high-resolution  projection  tech- 
nology.  Clearly,  a  CRT-based  projector  system 
could  not  provide  this  level  of  resolution. 

An  alternate  emerging  technology  for  providing 
such  resolution  is  laser-based  projection.  These  dis¬ 
plays  are  known  as  direct-write  systems  in  that  small, 
solid-state,  eye-safe  lasers  are  modulated  and  then 
scanned  direcdy  onto  a  screen.  No  display  phos¬ 
phors  or  LCD  elements  are  involved.  Early  proto¬ 
types  have  demonstrated  many  benefits  that  could 
substantially  improve  DMT  visual  displays.  Most 
importantly,  laser-based  displays  can  achieve  the 
spot  size  required  for  very  high  resolutions.  Other 
benefits  include  improved  display  contrast,  excellent 
display  brightness  (that  could  approach  daylight),  a 
significandy  improved  color  gamut,  and  a  large 
depth  of  focus.  Newly  developed  red,  green,  and 
blue  solid-state  laser  sources,  each  operating  at  a  pri¬ 
mary  color,  provide  the  most  efficient,  high-bright¬ 
ness  projection  light  source  for  display  applications 
to  date.  The  combination  of  solid-state  lasers,  paral¬ 
lel-scan  architectures,  and  new  high-speed  spatial 
light  modulators  that  modulate  the  video  into  the 
laser  beams,  make  a  5120  x  4096,  60  Hz,  noninter¬ 
laced  projector  possible.  This  projection  technology 
also  shows  promise  for  developing  lightweight,  high- 
resolution  helmet-mounted  displays  in  mobile,  vir¬ 
tual  task-training  platforms. 

Other  new  technologies,  such  as  commercial  PC- 
based  graphics,  could  be  used  to  improve  IG  capa¬ 
bility  and  at  the  same  time  drive  down  cost. 
PC-based  3D  graphics  have  made  significant  strides 
and  now  accelerate  many  aspects  of  graphical  visual¬ 
ization.  Competition  is  driving  advancements  at  a 
rapid  rate.  In  some  cases,  the  graphics  capabilities  of 


PC-driven  systems  are  nearly  doubling  every  six 
months.  While  these  are  some  of  the  current  solu¬ 
tions,  the  real  challenge  is  to  capitalize  on  these  new 
technologies  and  develop  super  high-resolution 
visual  system  components  that  include  super  high- 
resolution  projectors  (5120  x  4096,  60  Hz,  nonin¬ 
terlaced  video),  lightweight  high-resolution  displays, 
and  the  associated  image  generators  with  parallel, 
high-speed  digital  interfaces. 

Conclusion 

DMT  is  a  demanding  virtual  training  environment 
that  requiring  advanced  levels  of  IG  and  display  tech¬ 
nology  are  still  developing.  Major  advances  in  the 
capabilities  of  visual  systems  such  as  resolution,  con¬ 
trast,  and  brightness  need  to  be  achieved.  Develop¬ 
ment  of  IG  databases  in  any  team-training  scenario  is 
time-consuming.  However,  the  silver  lining  is  that 
new  technologies  are  rapidly  emerging  to  produce 
giant  leaps  in  these  directions.  While  microlaser  sys¬ 
tems  are  revolutionizing  the  high-end  IG  systems, 
commercial  PC-based  graphics  will  soon  accelerate 
DMT  visual  systems  with  gready  reduced  costs.  B 
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An  experimental  training  program  tests  the 
DMT  mettle  through  some  tough  terrain. 


IMPLEMENTING 

DISTRIBUTED 


MISSION 

TRAINING 


The  goal  of  Distributed  Mission  Training  is  to  provide  individuals 


with  opportunities  to  train  for  real-world  operational  missions  in  an  environment  that  is 
not  constrained  by  security,  cost,  and  safety  restrictions.  Even  the  most  elaborate  real- 
world  field  exercises  place  severe  limitations  on  participants  so  they  cannot  really  train  the 
way  they  will  operate.  Unlike  a  training  range,  pilots  in  a  virtual  flight  training  environ¬ 
ment  can  fly,  attack,  and  defend  themselves  as  they  would  in  wartime.  More  importandy, 
they  can  then  review  their  actions,  discuss  what  they  did  well  or  poorly  during  training, 
decide  how  they  would  do  it  differendy  in  the  future,  and  then  practice  new  plans  in  sub¬ 


sequent  training  events. 


Here,  we  present  a  case  analysis  of  a  DMT 
implementation.  An  experimental  training  pro¬ 
gram  named  RoadRunner’98  was  conducted  at 
the  U.S.  Air  Force  Research  Laboratory  to  evalu¬ 
ate  the  training  effectiveness  of  DMT  systems. 
RoadRunner’98  combined  virtual  (man-in-the- 
loop)  training  events  on  DMT  platforms  with 


computer-generated  constructive  models.  The 
systems  were  integrated  over  a  wide  geographic 
area  through  secure  networks  carrying  voice, 
data,  and  image  information. 

The  objectives  of  RoadRunner’98  were  to 
demonstrate  the  state-of-the-art  DMT  technolo¬ 
gies,  identify  the  strengths  and  shortfalls  of  these 
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technologies,  explore  how  to  best  use  this  new  training 
environment  to  enhance  team  training  and  develop  a 
research  and  development  agenda  for  die  future. 

DMT  Structure  of  Roadrunner’98 

RoadRunner’98  consisted  of  four  F-16,  one  A- 10, 
four  F-15,  and  AWACS  command  and  control  vir¬ 
tual  simulators  networked  both  locally  and  over  a 
wide  area  as  shown  in  Figure  1.  In  addition,  limited 
fidelity  platforms  were  linked  as  adversaries,  and  an 
observation  capability  was  provided  at  the  Pentagon. 
The  composite  force  missions  were  carried  out  over 
a  virtual  gaming  area.  Each  mission  was  executed 
over  a  secure,  wide-area  network  using  Distributed 
Interactive  Simulation  (DIS)  communications  pro¬ 
tocols.  In  these  missions,  virtual  platform  players 
interacted  with  constructive  forces  including 
friendly  and  enemy  fighters,  helicopters,  and  ground 
vehicles,  plus  enemy  surface-to-air  threats. 

All  links  in  the  network  were  commercial  T-l  lines 
accept  for  the  connection  from  Kirtland  Air  Force 
Base,  NM,  to  Tinker  Air  Force  Base,  OK,  which  was 
built  around  two  conventional  telephone  lines,  and 
the  connection  from  Mesa  to  Ft.  Dodge  was  an  exper¬ 
imental  satellite  link.  The  bandwidth  provided  by  the 
T-l  line  was  fully  sufficient  for  the  exercises  con¬ 
ducted.  However,  phone  line  connections  to  Tinker 
provided  significantly  less  bandwidth.  Therefore,  the 
data  stream  was  filtered  down  to  only  voice  commu¬ 
nication  and  aircraft  location.  While  this  bandwidth 
was  sufficient  to  support  virtual  AWACS  participa¬ 


tion,  the  connection  suffered  from 
reliability  problems. 

All  training  platforms  were  virtual. 
The  virtual  F-16  platforms  were 
equipped  with  the  Modular-Mobile 
Display  for  Advanced  Research  and 
Training  (M2DART) — a  state-of- 
the-art,  full-field  view,  rear-projec¬ 
tion,  dome-display  system.  The 
visual  imagery  was  provided  to  two  of 
the  F-16  platforms  from  Lockheed- 
Martin  SE2000+  computer  image 
generators,  which  utilized  polygo- 
nized  terrain  and  feature  representa¬ 
tions  augmented  with  cell  texture 
maps.  Imagery  was  provided  to  the 
other  platforms  from  Silicon  Graph¬ 
ics’  Reality  Monster  image  generators 
which  used  polygonized  terrain  rep¬ 
resentations  augmented  with  aerial 
mapping  photographs  of  the  training 
area.  Compared  to  the  Reality  Mon¬ 
sters,  the  SE2000+  image  generators 
provide  more  vertical  features  that  can  be  discerned  at 
low  altitude,  while  the  Reality  Monsters  provided 
more  scene  realism  at  medium  altitudes.  The  four  vir¬ 
tual  F-15  platforms  were  equipped  with  a  single  48- 
inch  CRT  displaying  each  forward,  out-the-window 
visual  imagery.  The  AWACS  controllers  used  a  train¬ 
ing  simulator  adapted  for  DMT  exercises. 

Constructive  simulations  provided  many  active 
entities,  including  AWACS  and  tanker  aircraft,  and 
supporting  friendly  combat  forces.  Constructive 
forces  also  included  enemy  fighter  aircraft  and  a  sur¬ 
face-to-air  defense  system  that  incorporate  radar,  mis¬ 
siles,  and  anti-aircraft  artillery.  The  control  consoles  at 
the  various  sites  provided  both  an  interface  for  system 
operators  to  conduct  a  mission  and  an  observation 
stadon  for  instructors  to  monitor  team  performance. 
The  control  console  incorporated  an  operators  sta¬ 
don,  the  test  directors  stadon,  and  an  observers  sta¬ 
tion.  The  instructor  monitored  mission  progress  on  a 
six-screen  video  display  system.  The  test  director 
could  communicate  with  all  participants  and  issue 
global  operational  commands.  The  post-mission 
analysis  systems  incorporated  synchronized  replay  of  a 
plan-view  display  together  with  videotapes  recorded 
from  each  virtual  training  platform. 

Training  Scenarios 

The  training  scenarios  were  based  on  team  exercise 
objectives,  technical  feasibility,  and  similarity  to 
F-16  and  F-15  operational  missions.  The  training 
scenarios  were  designed  to  simulate  a  small  portion 
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of  an  ongoing  conflict  that  included  both  ground 
and  air  forces.  In  these  scenarios,  F-l  6s  would  typi¬ 
cally  fly  into  enemy  controlled  territory  to  attack  a 
target  such  as  an  airfield  defended  by  both  surface 
and  airborne  threats  with  air  cover  from  the  F-l 5s. 
The  exercises  were  conducted  over  a  synthetic  gam¬ 
ing  area  in  Nevada.  Pilots  were  free  to  fly  anywhere 
within  the  gaming  area,  at  any  altitude,  and  at  any 
speed.  Pilots  were  also  free  to  employ  countermea¬ 
sures  against  missiles  as  required.  The  mission  sce¬ 
narios  required  pilots  to  fly  specified  routes  to  and 
from  the  target  areas,  which  were  selected  to  insure 
pilots  would  encounter  constructive  forces.  If 
attacked,  however,  pilots  were  free  to  deviate  from 
planned  routes.  The  role  of  the  virtual  F-l  5s  was  to 
achieve  air  superiority  within  a  specified  time  inter¬ 
val.  The  role  of  the  virtual  F-l 6s  was  air-to-surface 
attack,  and  the  task  of  the  AWACS  team  was  to  pro¬ 
vide  radar  coverage.  The  lead  F-l 6  pilot  was  the  mis¬ 
sion  commander  whose  tasks  were  to  organize  and 
instruct  the  different  virtual  teams  on  the  mission 
plan,  lead  and  coordinate  their  actions,  and  conduct 
post-mission  discussions. 

Three  sets  of  pilots  and  AWACS  crews  with 
300-1,200  actual  flying  hours  to  their  credit  partici¬ 
pated  in  the  study.  At  the  beginning  of  each  mission¬ 
planning  period,  the  mission  leader  contacted  the  other 
players  to  review  the  mission  plan.  At  the  scheduled 
time,  pilots  and  controllers  would  get  into  their  plat¬ 
forms  and  perform  systems  and  communication 
checks.  When  all  players  had  checked  in,  the  test  direc¬ 
tor  commenced  the  training  exercises.  After  each  sce¬ 
nario  was  completed,  teams  were  instructed  to  conduct 
their  post-mission  analyses  in  two  phases:  local 
(intrateam)  and  wide-area  (interteam).  During  local 
analysis,  teams  would  replay  the  mission  tapes  with  the 
goal  of  understanding  what  happened  during  the  mis¬ 
sion  and  generating  points  for  review  by  all  partici¬ 
pants.  The  teams  prepared  a  list  of  discussion  topics  for 
the  mission  commander  who  prepared  the  interteam 
discussion  agenda.  All  teams  would  then  restart  their 
tapes  at  the  beginning  of  the  mission  and  pause  for  dis¬ 
cussion  as  directed  by  the  mission  commander  At  the 
conclusion,  the  participants  filled  out  a  questionnaire 
identifying  performance  issues  of  DMT. 


Results  and  Mission  Performance 

RoadRunner’98  was  conducted  over  a  period  of  five 
days,  with  each  team  performing  two  training  mis¬ 
sions  per  day.  In  order  to  evaluate  the  training  effec¬ 
tiveness  of  DMT,  similar  missions  were  performed 
on  the  first  and  last  days  of  training.  This  exercise 
structure  enabled  an  assessment  of  team  perfor¬ 
mance  changes  attributable  to  the  DMT  experience 
in  the  five  days  of  training. 

All  missions  were  completed  with  no  aborts  or  sig¬ 
nificant  delays.  The  exercises  established  the  reliabil¬ 
ity  of  the  communication  systems  and  the  sufficiency 
of  the  bandwidth  provided  for  the  scenarios  tested. 
Comparing  team  performance  from  the  first  mission 
to  the  last  shows  that  accuracy  in  ordnance  delivery 
increased  by  30%  and  aircraft  losses  to  enemy  fighters 
decreased  by  44%.  ^ 


COMHU»*CATK>t«<»THEACM  Septemtwr 


l»»»/Vol.«2.No.9  93 


In  addition  to  mission  performance  data,  training 
instructors  observing  from  the  test  directors  console 
rated  their  team  using  a  (1-5)  teamwork  effectiveness 
scale.  Instructor  ratings  significandy  improved  for 
overall  mission  performance,  communications  and 
coordination,  situation  awareness,  and  crew  coordina¬ 
tion  from  the  first  day  to  the  last  day  of  training  as 
shown  in  Figure  2. 

Assessments  of  the  quality  of  DMT  were  carried 
out  by  the  trainees  as  well.  Before  the  start  of  the 
RoadRunner’98  program,  they  were  asked  how  well 
their  current  level  of  training  on  actual  aircraft  pre¬ 
pared  them  for  many  of  the  mission  tasks.  They  com¬ 
pleted  a  similar  questionnaire  at  the  end  of  the  week 
regarding  the  effectiveness  of  DMT  for  the  same 
tasks.  Aircraft  training  was  rated  higher  than  DMT 
primarily  for  tasks  involving  visual  detection  and 
recognition  of  aircraft  in  their  fields  of  view.  DMT 
was  rated  as  more  effective  than  aircraft  training  for 
tasks  infrequently  practiced  in  the  air  due  to  resource, 
safety,  and  security  considerations.  In  particular, 
DMT  was  rated  as  more  effective  for  radar  and  com¬ 
munications  intensive,  beyond-visual-range  tasks. 

Pilots  were  also  asked  to  critique  the  effectiveness 
of  DMT  after  each  mission.  The  weak  points  were 
the  inability  to  determine  another  players  aspects  at 
realistic  ranges  and  the  behaviors  of  some  of  the  con¬ 
structive  models.  The  consistent  strengths  of  DMT 
were  the  mission  recording  and  replay  systems,  the 
opportunities  for  premission  planning  and  post-mis¬ 
sion  analyses,  the  high  level  of  immersion  provided  by 
the  DMT  environment  compared  to  other  simula¬ 
tors,  and  the  instructors  ability  to  monitor  all  team 
players  from  the  console. 

Summary  and  Conclusions 

RoadRunner’98  and  similar  exercises  provide  infor¬ 
mation  that  will  significantly  impact  the  develop¬ 
ment  of  DMT  systems  and  applications.  Don 
Norman  [1],  in  his  work  on  product  development, 
describes  how  systems  designs  based  on  lists  of 
desired  features  often  fail  when  they  are  integrated 
into  work  environments.  He  recommends  that 
designs  should  emerge  from  observing  people  at 
work  along  with  frequent  prototype  evaluations  by 
users  and  the  design  team. 

Reviewing  participant  critiques  from  Road- 
Runner’98  demonstrates  the  value  of  Norman’s 
advice.  Everyone  who  has  flown  an  F-16/M2DART 
simulator,  other  than  fighter  pilots,  rates  its  fidelity 
and  training  potential  as  unlimited.  While  the  pilots 
in  RoadRunner’98  appreciated  the  scene  quality,  they 
also  criticized  simulators  for  foiling  to  display  other 
aircraft  adequately.  As  configured  during  Road¬ 


Runner’98,  the  F-16/M2DART  did  not  support  the 
intended  user’s  needs  although  non-pilots  would 
never  have  identified  this  shortcoming.  On  the  other 
hand,  AFRL!s  mission  replay  system,  which  was  a 
conglomeration  of  a  DIS  plan-view  display  and  four 
videotapes,  appears  awkward  and  clumsy  to  any  engi¬ 
neer.  While  pilots  recognized  its  flaws,  they  rated  its 
training  value  extremely  high.  Teams  would  fre¬ 
quently  spend  two  hours  reviewing  a  45-minute  mis¬ 
sion  and  still  be  talking  about  it  as  they  left  the 
debriefing  room.  During  RoadRunner’98,  the  replay 
system  was  neither  graceful  nor  elegant,  but  it  was 
highly  effective  in  providing  the  pilots  with  the  infor¬ 
mation  they  needed. 

The  objective  of  RoadRunner’98  was  to  evaluate 
the  quality  of  virtual  team  training  provided  with 
state-of-the-art  VR  and  networking  technologies  and 
to  evaluate  their  potential  for  significantly  enhanced 
training  capabilities.  Overall,  the  participants  and 
training  instructors  rated  DMT  as  very  effective 
within  specified  domains,  notably,  multiaircraft  air 
combat  against  multiple  enemy  fighters.  The  most 
significant  opportunities  for  improvement  in  DMT 
technologies  were  in  visual  displays,  constructive 
modeling,  simulation  of  electronic  interactions,  and 
systems  for  distributed  mission  planning  and  post¬ 
mission  analysis. 

Another  major  application  of  DMT  identified  by 
the  participants  and  instructors  is  to  use  these  tech¬ 
nologies  to  gain  leadership  skills  and  experience.  The 
team  leader  plans  the  mission,  makes  decisions, 
informs  the  other  players,  manages  the  missions,  and 
leads  the  post-mission  analyses.  Gaining  this  experi¬ 
ence  in  actual  aircraft  or  other  equipment  can  be  very 
expensive.  The  participating  players  in  Road¬ 
Runner’98  used  the  exercise  to  provide  leadership 
experience  for  younger  individuals  at  greatly  reduced 
cost  compared  to  aircraft  training.  Overall,  Road¬ 
Runner’98  demonstrated  both  the  value  and  limita¬ 
tions  of  DMT  and  also  illustrated  the  areas  for 
improvement  in  future  DMT  systems.  H 
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